An Efficient Model for Dynamic and Constrained Resource Allocation Problems

Dynamic constraint satisfaction is a useful tool for representing and solving sequential decision problems with complete knowledge in dynamic world and particularly constrained resource allocation problems. However, when resources are unreliable, this framework becomes limited due to the stochastic outcomes of the assignments chosen. On the contrary, Markov Decision Processes (MDPs) handle stochastic outcomes of unreliable actions, but their complexity explodes when using state-defined constraints. We thus propose an extension of the MDP framework so as to represent constrained and stochastic actions in sequential decision making. The basis of this extension consists in modeling the evolution of a dynamic constraint network by a MDP. We first study the complexity of the problem of finding an optimal policy for this model and then we propose an algorithm for solving it. Comparison to standard MDP shows that this framework noticeably improves policy computation.